Articles
the 9 marksarticlesaudiopublicationreviewsreading listchurch search
about usdonateeventseventscontact ussite maphome
Contributors





Jonathan Leeman

Should We Use Statistics to Measure Ministry?

By Jonathan Leeman

God's PottersA Review Article: God’s Potters: Pastoral Leadership and the Shaping of Congregations, by Jackson W. Carroll
Eerdmans, 2006, 296pp

Turn to the acknowledgment’s section of political scientist Robert Putnam’s book Bowling Alone and you’ll find about four thousand research assistants mentioned—including me. Yes, if you have a copy of the book, I’d be happy to sign it. Hold the applause, please.

For one year in the mid-nineties, I sat in the Library of Congress and other archives in the Washington DC area, looking for decade-by-decade membership data on multiple clubs, associations, or civic organizations over the last 150 years in a twenty city sample—groups like the Peoria Main-Street Presbyterian Church; the Galveston Austrian Benevolent Association; the Lathers, Wood, Wire, and Metal Union Local No. 68 (Denver); or the St. Lawrence O’Tool’s No. 32 Lodge of the Catholic Knights and Ladies of America (St. Louis). My job was to find this data, punch in the numbers cell by cell on a spreadsheet, and then email it to Putnam and his team, who processed it along with scores of figures from other political-science graduate-student lackeys.

I don’t know if any of the data I found "made it" into the book Bowling Alone or not. Yet it was this kind of statistical data that provided the foundation for Professor Putnam’s argument: Americans are less involved in civic organizations than ever before—from Masons, to Boy Scouts, to bowling leagues. The problem is, individuals learn how to be democratic citizens in such voluntary institutions. They learn how to cooperate, persuade, vote, and discuss their disagreements peaceably.

Fascinating stuff, I think. And Putnam’s ability to marshal statistical "evidence" for his thesis gives life to the argument.

Putnam’s work represents the love affair with statistics that became prevalent in social science departments in the latter half of the twentieth century. As the acids of postmodernism ate away at the foundations of so much philosophy, sociology, and historiography, many social scientists sought refuge (certitude) in the tools of the so-called hard sciences. The sea change was marked when a number of rational choice theorists, like University of Rochester political scientist William Riker or his student Kenneth Shepsle, now at Harvard with Putnam, were elected to the National Academy of Science. It’s been said that when the generation of disciples who followed Riker began to fill top universities, the American Political Science Review began to look like a physics journal, filled with pages of calculus equations, charts, and regression analyses.[1]

Behind this love affair with numbers and equations are behaviorist assumptions that go back in time further still. Statistical and rational choice analyses represent the most evolved stage of a brand of social science that tries to explain, measure, and predict human behavior according to rational considerations, much like the bio-chemical diagnoses of today’s lab-coated psychiatrists represent the most evolved and quantifiable stage of a way of thinking begun by the bespectacled Sigmund Freud, serenely leaning forward in his chair and asking patients about their dreams.

FROM THE SOCIAL SCIENCE SHELF TO THE CHURCH SHELF

The year after I worked for Putnam, I was called to ministry and lost my aspirations to enter the academic world of political science. What surprises me to this day, however, is picking up a book in the "theology" or "church" section of the bookstore, and finding the same old secular tools being used but now for spiritual purposes. Such is the case with the book God’s Potters: Pastoral Leadership and the Shaping of Congregations, by Jackson Carroll, a Methodist minister and Duke Divinity School professor.

God’s Potters is one of seven books that has emerged from Duke Divinity School’s massive research project on pastoral leadership in the United States, Pulpit & Pew (P&P). This volume presents all the findings of the P&P project in page after page of charts and graphs that give the reader a picture of ministerial life today. Throughout, Carroll writes from his "perspective as a social scientist and also an ordained minister" (x).

The data come from several sources: (i) a telephone survey of 1,231 senior or solo pastors conducted for P&P in 2001 by the University of Chicago; (ii) twenty-three focus groups of a diverse set of pastors; and (iii) the U.S. Congregational Life Survey of pastoral leaders in 434 congregations. Survey findings are grouped into four traditions: Roman Catholic, mainline Protestant, conservative Protestant or evangelical, and historic black churches. (On the whole, this ordained United Methodist minister does a good job of hiding his cards, though certain mainline Protestant facial tics do occur whenever the topics of women’s ordination or homosexuality come up.)

The surveys cover a lot of territory. Here are several findings I found interesting:

  • Women comprise zero percent of Catholic clergy, one percent of evangelical clergy, three percent of historic black clergy, and twenty percent of mainline Protestant clergy.
  • The Catholic church has the least number of clergy under age 45 (16 percent); evangelicals have the most (29 percent).
  • Pentecostal and "Independent Protestant" congregations are the most likely to agree their pastor is a good match for the congregation, while Methodists and Presbyterian Reformed are the least likely.
  • Mainline Protestant churches pay their pastors the most generous incomes in every category of church size, except for mega (1000+), for which no statistic is listed (?!).
  • Catholic and historic black clergy average the most working hours a week (55 and 54 hours, respectively), while mainline and conservatives average the least (48 and 47 hours).
  • Catholic clergy spend their largest segment of weekly time in "worship leadership (incl. preparation)" and administration—9 hours.
  • Mainline, conservative, and black Protestant clergy spend their largest bulk in "preaching (incl. preparation)"—all three average 10 hours per week.
  • Catholics, mainline Protestants, and historic black churches read what I would call the heavier stuff, while conservatives prefer the lighter stuff. Here are the three "most read" authors for each category:
    • Catholic—Henri J. M. Nouwen, John Paul II, Raymond Brown.
    • Mainline Protestant—Henri J. M. Nouwen, William Willimon, Frederick Buechner.
    • Historic black—Warren Wiersbe, John MacArthur, Matthew Henry.
    • And conservative Protestant—Max Lucado, John Maxwell, Charles Swindoll.
  • Pastors of every age, every church size, every gender, and every denominational tradition ranked "preaching," "teaching people about faith," and "worship leadership" as what they did best.
  • Mainline Protestant congregations ranked "conducting worship" as the most important thing pastors do, while conservative congregations pointed to "teaching about faith."

Once again, all this is fascinating. Statistics, when they can be trusted, give us a glimpse into what people value, where their priorities lie, what they believe. Here are two more revealing examples:

  • Two other surveys of how clergy spend their time, conducted in 1934 and 1954 among mainline Protestants, allow past clergy to be compared with present.
    • In 1934, the average mainline Protestant clergyman spent 22.5 hours per week on sermon preparation and delivery.
    • In 1954, he spent 8.5 hours.
    • In 2001, he or she spent 10.3 hours.
      What’s ministerial time being devoted to instead? Nothing, apparently.
    • In 1934, the average clergy worked 75.7 hours per week total.
    • In 1954, the figure dropped to 66.7 hours total.
    • In 2001, 50.8 hours total.
      Are pastors today lazier? Carroll says they are "more balanced." (We do need to be careful drawing strong conclusions from statistics of this kind in that social pressures of the 1930s and 50s would have been very different, potentially prompting pastors to count different things as work that pastors don’t today.)
  • Pastors were asked whether, when deciding on a new program or ministry, they would more likely consider the "theological rationale" for the change or "how well it meets the desires and needs of members or prospective members." Carroll reports that only 27 percent would give priority to the theological rationale, while 73 percent would give priority to how well member needs and desires were met. Moreover, differences between denominational traditions were not statistically significant, except for mainline Protestants who were "somewhat less likely" than others to consider a theological rationale. Nor were there significant differences when comparisons were made for church size, the pastor’s education level, or leadership style.

FROM THE STATISTICAL TO THE THEOLOGICAL

Since the results of these surveys appear to have provided the occasion for writing this book, most of my comments will respond to these. Yet we need to glance at least cursorily at the book’s larger storyline. The data are reported in chapters 3 to 6. On either side of these central chapters, Carroll devotes the opening and the closing chapters (1-2 and 7-8) to defining what a minister is, and what constitutes an excellent ministry. When this entire sandwich is considered together, the book begins to feel like Robert Putnam meeting George Lindbeck or Stanley Hauerwas.

Producers of Culture

Like Lindbeck or Hauerwas, Carroll describes the work and life of the church entirely in the language of cultural sociology, and a sociology in which moral and spiritual authority rests in a substantial way—I think (Carroll does not show us all his theological cards)—with the community. Ministers are the "culture producers"—God’s potters. They produce or give shape to the culture. They don’t do such work alone; they work with the members of the congregation, called "meaning makers." And all of them together "create meaning" out of the various "cultural objects" the church has been given, objects like the Bible or hymns or other church traditions. Moreover, each church creates meaning within their particular "social world."

The idea of ministers as "potters" does have a biblical aura. Carroll draws it from the apostle Paul’s reference to having treasure in "clay jars so that it may be clear that the extraordinary power belongs to God and does not come from us" (2 Cor. 4:7). Yet he has to make two inferential leaps to get where he wants to go: from the apostle as clay jar to congregation as clay jar; and from congregation as clay jar to ministers as potters.

Now a writer is not bound to use an explicitly scriptural metaphor—like shepherd—for describing the work of ministers. And Scripture uses a number of metaphors to describe a minister’s work because so much is involved in his work—prophetic aspects, ruling aspects, mediating aspects, and so forth. From the standpoint of a biblical worldview, however, the choice of "potter" does strike me as strange. In doing theology, giving primacy to a secondary or, as here, an inferential metaphor often distorts Scripture’s intended meaning. Classic liberals do this, for instance, with theories of the atonement. Open theists do this with the doctrine of God’s knowledge. In both cases proof texts are overemphasized or over-inference-ized and the forest is lost for the trees. When it comes to the Bible’s theology of leadership, perhaps it’s not incidental that someone who gives primacy to the work of leaders and churches as "creating meaning" out of "cultural objects" like the Bible should choose a metaphor that the authors of Scripture, interestingly, reserve for God himself (Gen. 2:7; Job 10:9; Is. 29:16; Jer. 18:6; Rom. 9:21), which is not true of other metaphors for leadership (again, like shepherd). Is God not the true potter and culture shaper? Biblically, one could say that ministry is culture-shaping, but only as quickly as one would say that an ambassador’s work is culture-shaping—it’s true, but it misses the main point. The point is representing someone else, and delivering someone else’s message (see 2 Cor. 5:20). The ambassador who grows tired of his charge to deliver a message, and who begins to fancy himself as a "Shaper of Culture," I fear, is an ambassador who will soon distort the message and manipulate opportunities for his own gain. So too with every minister who does not recognize that his ministry is stewardship, his authority representative, his message not his own, and his life mediatorial (see 2 Cor. 4:2). This is not to deny the unique ways God may gift a certain man "to bring to the table" his own personality and gifts in a particular context; it’s simply an attempt to place our emphases in the right places. Distorted emphases lead to distorted and manipulative ministries.

Producers of High Culture

With these sociological (communitarian? postliberal?) definitions set in place in chapters 1 and 2, Carroll then re-employs them in the final chapters on ministerial excellence (7 and 8). What is an excellent ministry? It’s ministry that produces not just culture, but which reproduces "high culture," a corporate life that will be defined variously between Catholics and Protestants, mainline and conservative, white and black. In other words, each tradition has a way of interpreting the Bible and church tradition in order to yield its conceptions of excellent ministry practices. Excellent ministry is then ministry that reproduces these excellent practices.

Commendably, Carroll sets out to avoid measuring excellence in ministry by the values of the business world, values like growth, hard work, efficiency, avoidance of weakness, and so forth. The high culture of a Christian church should be much more "cruciform," he says, that is, more dedicated to showing God’s power in the life, death, and resurrection of Jesus through the church’s vulnerability, service, hope-giving message, and love. In the end, however, it’s difficult to see how Carroll’s "strategies for excellence" would not be satisfying to the business world:

  • "recruit for excellence" by creating a culture of call in churches;
  • "educate for excellence" by encouraging more study and continued study among ministers;
  • "congregationally and denominationally support excellence" by paying ministers more;
  • and "taking responsibility for excellence" among ministers themselves by practicing spiritual disciplines, developing their imagination, nurturing holy friendships, maintaining boundaries between work and family life, and being diligent in caring for their own physical and emotional well-being.

In short, Carroll calls churches and ministers to recruit better, educate better, pay better, and self-motivate better.

FROM SOCIOLOGY TO SUI GENERIS

If I were reviewing Carroll’s book as a pure secularist, I might try to deconstruct the argument by exposing the tension between the postmodern and modern—between the epistemic priority given first to the community’s high culture and then to statistics.

If I were reviewing Carroll’s book as a businessman, I might try to demonstrate that, in the end, his solutions are just as pragmatic as what I might find in any business journal.

If I were reviewing Carroll’s book as theologian, I might want to have a discussion about defining ministerial excellence according to high culture instead of Scripture, as well as about letting one’s sociology shape one’s hermeneutic more than having Scripture shape one’s hermeneutic and sociology.

Yet I want to go in a related, but slightly different direction: what should a minister make of all these statistics? After all, it’s the P&P project which prompted the book in the first place. What should pastors and church leaders take from all these surveys, or the surveys of George Barna, Thom Rainer, or others?

Why Statistics Are Useful

Statistics have a way of making abstract claims concrete, of reinforcing convictions, or of undermining long-held assumptions. In that regard, statistics are very useful. Conservatives claim to believe in the authority and sufficiency of God’s Word. So why do they spend no more time preparing their sermons than mainline Protestants? If they really believe the Bible, why don’t they spend more time studying it?

Most ministers probably believe they work hard. Do they? Compared to ministers in 1934?

We could keep going, but the point should be clear. Like balancing the checkbook, hard numbers can quickly expose habits, values, and decision-making patterns that are taken for granted. Do you consider yourself a generous person? What would your bank statement say?

Why Statistics Can Be Misleading

But surely the utility of numbers is limited at best, deceiving at worst. Does a large church mean that the preaching has been sound…or entertaining? Does less time in the study in preparation for a sermon mean less care for Scripture…or more time stuck in a daily commute…or longer hours on the knees praying? Who knows! The real issue, and the reason I’m always surprised to find books in the "church" section reading like books in the political science section, is this: how can we quantify the movement of the supernatural? How accurately can we really evaluate those things which the Bible assures us can only be seen with eyes of faith? How well can we discern what’s in the mind of God?

An analogy here might be useful, one that occurred to me while reading Mark Thompson’s recent monograph on the clarity of Scripture. Thompson observes that many writers approach the interpretation of Scripture like they approach the interpretation of any other text, submitting the biblical text to ordinary rules of grammar, genre, structure, syntax, and literary devises. In so far as Scripture has been written by men, this is entirely appropriate and yields valuable fruit. But for those who also affirm that Scripture has been written by God, such tools are finally not enough.

The relationship between author and text is not the same as in other texts. The relationship between author and reader (any reader) is different in the case of the Bible to any other text. In other words, the sui generis character of this text needs to be respected…There is much to appreciate in the way in which Scripture has been constructed as literature. Yet neither historical nor literary analysis is sufficient when it comes to the stuff of the Bible…Christian engagement with the Bible cannot forget, ignore, or suspend for the sake of some higher goal, the identification of this text as the written word of God (A Clear and Present Word, 135-36).

I had to look up sui generis in the dictionary when I read it. It means "constituting a class alone: unique, peculiar." Now, the church on earth offers nothing as infallible or authoritative as the word of God. Yet among the organizations that fall within Robert Putnam’s research bailiwick, surely, Christians would affirm that the church constitutes a class alone. God has not elected the Boy Scouts. Christ did not die for a Masonic Lodge. The Holy Spirit does not regenerate and sanctify bowling leagues (at least in their capacity as bowling leagues).

In other words, the very things that give life and breath to the church cannot be seen or measured.

A hundred Boy Scouts can meet in a room, as can a hundred Masons, as can a hundred Muslims, as can a hundred people calling themselves "Christian." What’s the difference between these groups of people? Statistically, nothing. What’s the difference between them spiritually? Hopefully, everything. But spiritual differences can only be seen with spiritual eyes. They cannot be surveyed with the kinds of questions human beings are capable of answering by checking a box, unless of course ministers and churches could answer questions like these:

  • What percentage of the conversions in your church over the last year were genuine?
  • What percentage of the people singing on Sunday were praising God in their hearts, and what percentage were playing along?
  • Is the recent numerical growth in your church a sign of God’s blessing and favor or merely the effectiveness of worldly devices?
  • Is God behind it or not?Is the recent lack of numerical growth in your church a result of (i) dated evangelistic methods, (ii) God’s decision in eternity past to close the doors of your church and give the city in which you presently live over to its sin, or (iii) God’s decision in eternity past to teach your church to faithfully persevere in the midst of opposition for several years before blessing it with visible fruit?
  • If you answered (iii) to the question above, then how many years does God intend for your church to persevere before it will witness outward fruit?

Our imaginary survey could go on for pages. I hope the reader gets the point. Statistics can be useful for churches. But the most important things about a church cannot be measured by the devices of sociology or statistics. And here I’m talking about the differences between fake and real, between flesh and spirit, between the minds of men and the mind of God.

How then should we measure excellence in ministers and churches? We should measure them entirely according to whether they are faithful to living by and proclaiming God’s Word. That’s the measurement Paul uses in his chapter on clay jars.

Have you taken the apostle’s survey found in 2 Corinthians 4? Take a moment now to fill it out:

  • Have you renounced secret and shameful ways or used deception in the work of the ministry (2 Cor. 4:2)?
  • Have you distorted the word of God or have you set forth the truth of God plainly, commending yourself to every man’s conscience in the sight of God (2 Cor. 4:2)?
    Remember that "the god of this age has blinded the minds of unbelievers, so that they cannot see the light of the gospel of the glory of Christ, who is the image of God" (2 Cor. 4:4). Human devices and standards of excellence will prove futile!
  • Do you preach yourself or Christ Jesus as Lord and yourself as servants for Jesus’ sake (2 Cor. 4:5)?
  • Would your fellow staff and members of your church be more likely to say, "He’s impressive!" or "He’s nothing more than a clay jar, but God sure seems to use him!" In other words, is your ministry marked by the display of human excellence, or by the display of an "all-surpassing divine power" amidst your weakness, your affliction, your being perplexed, your persecution, your being struck down, your carrying around in your body the death of Jesus with the foolish message of the gospel (2 Cor. 4:7-10; 1 Cor. 1:17-19)?

May God indeed grant you and me the cruciform humility to pursue such "excellent" ministries!

[1] See Jonathan Cohn, "Irrational Exhuberance," The New Republic, Oct. 25, 1999: 25-31.

Jonathan Leeman is the director of communications for 9Marks.

March 2007
©9Marks

Permissions: You are permitted and encouraged to reproduce and distribute this material in any format, provided that you do not alter the wording in any way, you do not charge a fee beyond the cost of reproduction, and you do not make more than 1,000 physical copies. For web posting, a link to this document on our website is preferred. Any exceptions to the above must be explicitly approved by 9Marks.

Please include the following statement on any distributed copy: ©9Marks. Website: www.9Mark.org.